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(57) Abstract: Onecmbodimcnl ol'lhc present invcniion provides a system thai marks 
mcmorv' elements based upon how informalion retrieved from the memory elements al'- 
Fecis speculative program execution. 'Phis system operates by allowing a programmer 
u> examine source code thai is to be compiled into executable code lor a head thread 
that executes program instructions, and for a speculative thread that executes program 
instructions in advance of the head thread. During read operations to memory ele- 
ments by the speculative thread, this executable code generally causes the speculative 
thread to update status information associated with the memory elements to indicate 
thai the memory elements have been read by Ihc speculative thread. Next, the system 
allows the programmer to idcnlily a given read operation directed to a given memory 
ciemcnl. wherein a given value retrieved From the given memory element during the 
given read operation does not alTecl subsequent execution of the speculative thread. 
The programmer is then allowed to insert a hint into the source code specifying that 
the speculative ihread is not lo update status informalion during the given read opera- 
tion directed lo the given memory elemeni. Next. Ihc system compiles the source code, 
including the hinu into the cxccuiablc code, so lhal during the given read operation, 
the cxccuiablc code does noi cause the speculative thread to updaie status information 
associated with Ihc given memory elemeni lo indicate that the given meni(»ry element 
has been read by Ihe speculative Ihread. 
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MARKING MEMORY ELEMENTS BASED UPON 
USAGE OF ACCESSED INFORMATION DURING 
SPECULATIVE EXECUTION 



BACKGROUND 

Field of the Invention 

10 The present invention relates to compilers and techniques for improving 

computer system performance. More specifically, the present invention relates to a 
method and an apparatus that supports marking a memory element based upon how 
information retrieved from the object is used in order to facilitate speculative program 
execution. 

15 

Related Art 

As increasing semiconductor integration densities allow more transistors to be 
integrated onto a microprocessor chip, computer designers are investigating different 
methods of using these transistors to increase computer system performance. Some 

20 recent computer architectures exploit "instruction level parallelism," in which a single 
central processing unit (CPU) issues multiple instructions in a single cycle. Given 
proper compiler support, instruction level parallelism has proven effective at 
increasing computational performance across a wide range of computational tasks. 
However, inter-instruction dependencies generally limit the performance gains 

25 realized from using instruction level parallelism to a factor of two or three. 

Another method for increasing computational speed is "speculative execution" 
in which a processor executes multiple branch paths simultaneously, or predicts a 
branch, so that the processor can continue executing witliout waiting for the result of 
the branch operation. By reducing dependencies on branch conditions, speculative 

30 execution can increase the total number of instructions issued. 
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Unfortunately, conventional speculative execution typically provides a 
limited performance improvement because only a small number of instructions can be 
speculatively executed. One reason for this limitation is that conventional speculative 
execution is typically performed at the basic block level, and basic blocks tend to 
5 include only a small number of instructions. Another reason is that conventional 
hardware structures used to perform speculative execution can only accommodate a 
small number of speculative instructions. 

What is needed is a method and apparatus that facilitates speculative execution 
of program instructions at a higher level of granularit}' so that many more instructions 

10 can be speculatively executed. 

One challenge in designing a system that supports speculative execution is to 
detect a rollback condition. A rollback condition can occur in a number of situations. 
For example, a rollback condition can occur when a speculative thread that is 
executing program instructions in advance of a head thread reads from a memory 

15 element before the head thread performs a store to the memory element. In this case, 
the speculative thread must "rollback" so that it can read the value stored by the head 
thread. A rollback condition can be detected by "marking" memory elements as they 
are read by the speculative thread so that the head thread can subsequently determine 
if the memory elements have been read by the speculative thread. 

20 However, not all data values that are read by a speculative thread affect 

subsequent execution of the speculative thread. For example, a lookup into a hash 
table by the speculative thread may involve a number of preliminary accesses to 
memory elements in the hash table before a desired hash table entry is Jocated. Note 
that these preliminary accesses do not affect subsequent actions of the speculative 

25 thread. Hence, marking these memory elements can cause future accesses to these 
memory elements to generate unnecessary rollbacks, which can reduce the 
performance gains arising from speculative execution. 

What is needed is a method and an apparatus for selectively marking accessed 
memory elements based upon whether information retrieved from the accessed 

30 memory elements affects subsequent actions of the speculative thread. 



wo (n/93028 



PCT/US(n/17645 



3 

SUMMARY 

One embodiment of the present invention provides a system that marks 
memory elements based upon how information retrieved from the memory elements 
affects speculative program execution. This system operates by allowing a 
5 programmer to examine source code that is to be compiled into executable code for a 
head thread that executes program instructions, and for a speculative thread that 
executes program instructions in advance of the head thread. During read operations 
to memory elements by the speculative thread, this executable code generally causes 
the speculative thread to update status information associated with the memory 

1 0 elements to indicate that the memory elements have been read by the speculative 
thread. Next, the system allows the programmer to identify a given read operation 
directed to a given memorj' element, wherein a given value retrieved from the given 
memory element during the given read operation does not affect subsequent execution 
of the speculative thread. The programmer is then allowed to insert a hint into the 

15 source code specifying that the speculative thread is not to update status information 
during the given read operation directed to the given memory element. Next, the 
system compiles the source code, including the hint, into the executable code, so that 
during the given read operation, the executable code does not cause the speculative 
thread to update status information associated with the given memory element to 

20 indicate that the given memory element has been read by the speculative thread. 

In one embodiment of the present invention, during a subsequent write 
operation to a memory element by the head thread, the executable code causes the 
head thread to check status information associated with the memory element to 
determine if the memory element has been read by the speculative thread. If the 

25 memory element has been read by the speculative thread, the system causes the 
speculative thread to roll back in order to read a result of the write operation. 

In one embodiment of the present invention, inserting the hint into the source 
code involves inserting a comment into the source code that causes a compiler to 
generate executable code for the speculative thread that does not to mark the given 

30 memory element during the given read operation. 
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In one embodiment of the present invention, compiling the source code into 
the executable code involves setting a look attribute associated with the given 
memory element to indicate that the speculative thread is not to mark the given 
memory element. Note that the code for marking (extra stores and loads) is not in the 
5 executable code. Hence, it results in increased speed of execution of the speculative 
thread. 

In one embodiment of the present invention, the executable code causes a join 
operation between the head thread and the speculative thread when the head tliread 
reaches a point in a program where the speculative thread began executing This join 
10 operation causes state associated with the speculative thread to be merged with state 
associated with the head thread. 

In one embodiment of the present invention, the system additionally integrates 
either the executable code or the source code into a system library. 

In one embodunent of the present invention, the memory elements include 
1 5 objects defined within an object-oriented programming system. 

BRIEF DESCRIPTION OF THE FIGURES 
FIG. 1 illustrates a computer system including two central processing units 
sharing a common data cache in accordance with an embodunent of the present 
20 invention. 

FIG. 2A illustrates sequential execution of methods by a single thread. 

FIG. 2B illustrates space and time dimensional execution of a method in 
accordance with an embodiment of the present invention. 

FIG. 3 illustrates the state of the system stack during space and time 
25 dimensional execution of a method in accordance with an embodiment of the present 
invention. 

FIG. 4 illustrates how memory is partitioned between stack and heap in 
accordance with an embodiment of the present invention. 

FIG. 5 illustrates the structure of a primary version and a space-time 
30 dimensioned version of an object in accordance with an embodiment of the present 
invention. 
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FIG. 6 illustrates the structure of a status word for an object in accordance 
with an embodiment of the present invention. 

FIG. 7 is a flow chart illustrating operations involved in performing a write to 
a memory element by a head thread in accordance with an embodiment of the present 
5 mvention. 

FIG. 8 is a flow chart illustrating operations involved in performing a read to a 
memory element by a speculative thread in accordance with an embodiment of the 
present invention. 

FIG. 9 is a flow chart illustrating operations involved in performing a write to 
1 0 a memory element by a speculative thread in accordance with an embodiment of the 
present invention. 

FIG. 10 is a flow chart illustrating operations involved in performing a join 
between a head thread and a speculative thread in accordance with an embodiment of 
the present invention. 

15 FIG. 1 1 is a flow chart illustrating operations involved in performing a join 

between a head thread and a speculative thread in accordance with another 
embodiment of the present invention. 

FIG. 12 is a flow chart illustrating the process of producing code so that a 
speculative thread does not mark selected memory elements in accordance with an 
20 embodiment of the present invention. 

FIG. 13 is a flow chart illustrating the process of compiling source code into 
executable code so that a speculative thread does not mark selected memory elements 
in accordance with an embodiment of the present invention. 

25 DETAILED DESCRIPTION 

The following description is presented to enable any person skilled in the art 
to make and use the invention, and is provided in the context of a particular 
application and its requirements. Various modifications to the disclosed embodiments 
will be readily apparent to those skilled in the art, and the general principles defined 
30 herein may be applied to other embodiments and applications without departing from 
the spirit and scope of the present invention. Thus, the present invention is not 
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intended to be limited to the embodiments shown, but is to be accorded the widest 
scope consistent with the principles and features disclosed herein. 

The data structures and code described in this detailed description are typically 
stored on a computer readable storage medium, which may be any device or medium 
that can store code and/or data for use by a computer system. This includes, but is not 
limited to, magnetic and optical storage devices such as disk drives, magnetic tape, 
CDs (compact discs) and DVDs (digital video discs), and computer instruction signals 
embodied in a carrier wave. For example, the carrier wave may carry information 
across a communications network, such as the Internet. 



Computer System 

FIG. 1 illustrates a computer system including two central processing units 
(CPUs) 102 and 104 sharing a common data cache 106 in accordance witli an 
embodiment of the present invention. In this embodiment, CPUs 102 and 104 and 

15 data cache 106 reside on silicon die 100. Note that CPUs 102 and 104 may generally 
be any type of computational devices that allow multiple threads to execute 
concurrently. In the embodiment illustrated in FIG. 1, CPUs 102 and 104 are very 
long instruction word (VLIW) CPUs, which support concurrent execution of multiple 
instructions executing on multiple functional units. VLIW CPUs 102 and 104 include 

20 instruction caches 112 and 120, respectively, containing instructions to be executed 
by VLIW CPUs 102 and 104. 

VLIW CPUs 102 and 104 additionally include load buffers 1 14 and 122 as 
well as store buffers 116 and 124 for buffering communications with data cache 106. 
More specifically, VLIW CPU 102 includes load buffer 114 for buffering loads 

25 received from data cache 106, and store buffer 116 for buffering stores to data cache 
106. Similarly, VLIW CPU 104 includes load buffer 122 for buffering loads received 
from data cache 106, and store buffer 124 for buffering stores to data cache 106. 

VLIW CPUs 102 and 104 are additionally coupled together by direct 
communication link 128, which facilitates rapid communication between VLIW 

30 CPUs 102 and 104. Note that direct communication link 128 allows VLIW CPU 102 
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to write into communication buffer 126 within VLIW CPU 104. It also allows 
VLIW CPU 104 to write into communication buffer 1 18 witliin VLIW CPU 102. 

In the embodiment illustrated in FIG. 1, Data cache 106 is fully dual-ported 
allowing concurrent read and/or write accesses from VLIW CPUs 102 and 104. This 
5 dual porting eliminates cache coherence delays associated with conventional shared 
memory architectures that rely on coherent caches. 

In one embodiment of the present invention, data cache 106 is a 16K byte 4- 
way set-associative data cache with 32 byte cache lines. 

Data cache 106, instruction caches 1 12 and instruction cache 120 are coupled 
1 0 through switch 1 1 0 to memory controller 111. Memory controller 1 11 is coupled to 
dynamic random access memory (DRAM) 108, which is located off chip. Switch 1 10 
may include any type of circuitry for switching signal lines. In one embodiment of 
the present invention, switch 1 10 is a cross bar switch. 

The present invention generally applies to any computer system that supports 
15 concurrent execution by multiple threads and is not limited to the illustrated 
computing system. However, note tliat data cache 106 supports fast accesses to 
shared data items. These fast accesses facilitate efficient sharing of status information 
between VLIW CPUs 102 and 104 to keep track of accesses to versions of memory 
objects. 

20 

Space-Time Dimensionat Execution of Methods 

FIG. 2A illustrates sequential execution of methods in a conventional 
computer system by a single head thread 202. In executing a program, head thread 
202 executes a number of methods in sequence, including method A 204, method B 
25 206 and method C 208. 

In contrast, FIG. 2B illustrates space and time dimensional execution of a 
method in accordance with an embodiment of the present invention. In FIG. 2B, head 
thread 202 first executes method A 204 and then executes method B 206. (For this 
example, assume that method B 206 returns a void or some other value that is not 
30 used by method C 208. Alternatively, if method C 208 uses a value returned by 
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method B206, assume that method C 208 uses a predicted return value from method 
B 206.) 

As head thread 202 executes method B 206, speculative thread 203 executes 
metliod C 208 in a separate space-time dimension of the heap. If head thread 202 
5 successfully executes method B 206, speculative thread 203 is joined with head thread 
202. This join operation involves causing state associated with the speculative thread 
203 to be merged with state associated with the head tliread 202 and the collapsing of 
the space-time dimensions of the heap. 

If speculative thread 203 for some reason encounters problems in executing 
10 method C 208, speculative thread 203 performs a rollback operation. This rollback 
operation allows speculative thread 203 to reattempt to execute metliod C 208. 
Alternatively, head thread 202 can execute method C 208 non-speculatively and 
speculative thread 203 can execute a subsequent method. 

There are a number of reasons why speculative thread 203 may encounter. 
15 problems in executing method C 208. One problem occurs when head thread 202 
executing method B 206 writes a value to a memory element (object) after speculative 
thread 203 has read the same memory element. The same memory element can be 
read when the two space-time dimensions of the heap are collapsed at this memory 
element at the time of the read by speculative thread 203. In this case, speculative 
20 tliread 203 should have read the value written by head thread 202, but instead has read 
a previous value. In this case, the system causes speculative thread 203 to roll back so 
that speculative thread 203 can read the value written by head thread 202. 

Note that the term "memory element" generally refers to any unit of memory 
that can be accessed by a computer program. For example, the term "memory 
25 element" may refer to a bit, a byte or a word memory, as well as a data structure or an 
object defined within an object-oriented programming system. 

FIG. 3 illustrates the state of the system stack during space and time 
dimensional execution of a method in accordance with an embodiment of the present 
invention. Note that since programming languages such as the Java programming 
30 language do not allow a method to modify the stack frame of another method, the 
system stack will generally be the same before method B 206 is executed as it is 
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before method C 208 is executed. (This is not quite true if method B 206 returns a 
parameter through the system staclc. However, return parameters are can be explicitly 
dealt with as is described below.) Referring the FIG. 3, staclc 300 contains method A 
frame 302 while method A 204 is executing. When method A 204 returns, method B 
5 206 commences and method A frame 302 is replaced by method B frame 304. 

Finally, when method B 206 returns, method C 208 commences and method B frame 
304 is replaced by method C frame 306. Note that since stack 300 is the same 
immediately before method B 206 executed as it is immediately before method C 208 
is executed, it is possible to execute method C 208 using a copy of stack 300 without 

10 first executing method B 206. 

In order to undo the results of speculatively executed operations, updates to 
memory need to be versioned. The overhead involved in versioning all updates to 
memory can be prohibitively expensive due to increased memory requirements, 
decreased cache performance and additional hardware required to perform the 

15 versionuig. 

Fortunately, not all updates to memory need to be versioned. For example, 
updates to local variables - such as a loop counter - on a system stack are typically 
only relevant to the thread that is updating the local variables. Hence, even for 
speculative threads versionmg updates to these local variables is not necessary. 

20 When executing programs written in conventional programming languages, 

such as C, it is typically not possible to determine which updates are related to the 
heap, and which updates are related to the system stack. These programs are typically 
compiled from a high-level language representation into executable code for a 
specific machine architecture. This compilation process typically removes 

25 distinctions between updates to he^ and system stack. 

The same is not true for new platform-independent computer languages, such 
as the JAVA™ programming language distributed by SUN Microsystems. Inc. of 
Palo Alto, California. (Sun, the Sun logo. Sun Microsystems, and Java are trademarks 
or registered trademarks of Sun Microsystems, Inc. in the United States and other 

30 countries.) A program written in the Java programming language is typically 
compiled into a class file containing Java byte codes. This class file can be 
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transmitted over a computer network to a distant computer system to be executed on 
the distant computer system. Java byte codes are said to be "platform-independent," 
because they can be executed across a vs'ide range of computing platforms, so long as 
the computing platforms provide a Java virtual machine. 
5 A Java byte code can be executed on a specific computing platform by using 

an interpreter or a just in time (JIT) compiler to translate the Java byte code into 
machine code for the specific computing platform. Alternatively, a Java byte code 
can be executed directly on a Java byte code engine running on the specific 
computing platform. 

10 Fortunately, a Java byte code contains more syntactic information than 

conventional machine code. In particular, the Java byte codes differentiate between 
accesses to local variables in the system stack and accesses to the system heap. 
Furthermore, programs written in the Java programming language do not allow 
conversion between primitive and reference types. Such conversion can make it hard 

15 to differentiate accesses to the system stack from accesses to the system heap at 
compile time. 

Data Structures to Support Space-Time Dimensional Execution 

FIG. 4 illustrates how memory is partitioned between stack and heap in 

20 accordance with an embodiment of tlie present invention. In FIG. 4, memory 400 is 
divided into a number of regions including heap 402, stacks for threads 404 and 
speculative heap 406. Heap 402 comprises a region of memory from which objects 
are allocated. Heap 402 is further divided into younger generation region 408 and 
older generation region 410 for garbage collection purposes. For performance 

25 reasons, garbage collectors typically treat younger generation objects differently from 
older generation objects. Stack for threads 404 comprises a region of memory from 
which stacks for various threads are allocated. Speculative heap 406 contains the 
space-time dimensioned values of all memory elements where the two space-time 
dimensions of the heap are not collapsed. This includes space-time dimensional 

30 versions of objects, for example, version 510 of object 500 as shown in FIG. 5, and 
objects created by speculative thread 203. For garbage collection purposes, these 
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objects created by speculative thread 203 can be treated as belonging to a generation 
that is younger than objects within younger generation region 408. 

FIG. 5 illustrates the structure of a primary version of object 500 and a space- 
time dimensioned version of object 510 in accordance with an embodiment of the 
5 present invention. 

Primary version of object 500 is referenced by object reference pointer 501. 
Like any object defined within an object-oriented programming system, primary 
version of object 500 includes data region 508, which includes one or more fields 
containing data associated with primary version of object 500. Primary version of 
10 object 500 also includes method vector table pointer 506. Method vector table pointer 
506 points to a table containing vectors that point to the methods that can be invoked 
on primary version of object 500. 

Primary version of object 500 also includes space-time dimensioned version pointer 
502, which points to space-time dimensioned version of object 510, if the two space- 

1 5 time dimensions are not collapsed at this object. Note that in the illustrated 

embodiment of the present invention, space-time dimensioned version 510 is always 
referenced indirectly through space-time dimensioned version pointer 502. Primary 
version of object 500 additionally includes status word 504, which contains status 
information specifying which fields from data region 508 have been written to or read 

20 by speculative thread 203 . Space-time dimensioned version of object 5 1 0 includes 
only data region 518. 

FIG, 6 illustrates the structure of status word 504 in accordance with an 
embodiment of the present invention. In this embodiment, status word 504 includes 
checkpoint number 602 and speculative bits 603. Speculative bits 603 includes read 

25 bits 604 and write bits 606. When status word 504 needs to be updated due to a read 
or a write by speculative thread 203, checkpoint number 602 is updated with the 
current time of the system. The current time in the time dimension of the system is 
advanced discretely at a join or a rollback. This allows checkpoint number 602 to be 
used as a qualifier for speculative bits 603. If checkpoint number 602 is less than the 

30 current time, speculative bits 603 can be interpreted as reset. 
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Read bits 604 keep track of which fields within data region 508 have been 
read since the last join or rollback. Correspondingly, write bits 606 keep track of 
which fields within data region 508 have been written since the last join or rollback. 
In one embodiment of the present invention, read bits 604 includes one bit for each 
5 field within data region 508. In another embodiment, read bhs includes fewer bits 
than the number of fields within data region 508. In this embodiment, each bit within 
read bits 604 corresponds to more than one field in data region 508. For example, if 
there are eight read bits, each bit corresponds to every eighth field. Write bits 606 
similarly can correspond to one or multiple fields within data region 508. 

10 

Space-Time Dimensional Update Process 

Space-time dimensioning occurs during selected memory updates. For local 
variable and operand accesses to the system stack, no space-time dimensioned 
versions exist and nothing special happens. During read operations by head thread 

15 202 to objects in the heap 402, again nothing special happens. 

Special operations are involved in write operations by head thread 202 as well 
as read and write operations by speculative thread 203. These special operations are 
described in more detail with reference to FIGs7, 8 and 9 below. 

FIG. 7 is a flow chart illustrating operations involved in a write operation to an 

20 object by a head thread 202 in accordance with an embodiment of the present 

invention. The system writes to the primar>' version of object 500 and the space-time 
dimensioned version of object 510 if the two space-time dimensions are not collapsed 
at this point (step 702). Next the system checks status word 504 within primary 
version of object 500 to determine whether a rollback is required (step 704). A 

25 rollback is required if speculative thread 203 previously read the data element. The 
same memory element can be read when the two space-time dimensions of the heap 
are collapsed at this memorj^ element at the time of the read by speculative thread 
203. A rollback is also required if speculative thread 203 previously wrote to the 
object and thus ensured that the two dimensions of the object are not collapsed at this 

30 element, and if the current write operation updates both primary version of object 500 
and space-time dimensioned version of object 510. 
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If a rollback is required, the system causes speculative thread 203 to perform 
a rollback operation (step 706). Tliis rollback operation allows speculative thread 203 
to read from (or write to) the object after head thread 202 writes to the object. 

Note that in the embodiment of the present invention illustrated in FIG. 7 the 
5 system performs writes to both primary version 500 and space-time dimensioned 
version 510. In an alternative embodiment, the system first checks to determine if 
speculative thread 203 previously wrote to space-time dimensioned version 510. If 
not, the system writes to both primary version 500 and space-time dimensioned 
version 510. If so, the system only writes to primary version 500. 
10 FIG. 8 is a flow chart illustrating operations involved in a read operation to an 

object by speculative thread 203 in accordance with an embodiment of the present 
invention. During this read operation, the system sets a status bit in status word 504 
within primary version of object 500 to indicate that primary version 500 has been 
read (step 802). Speculative thread 203 then reads space-time dimensioned version 
15 5 1 0, if it exists. Otherwise, speculative thread 203 reads primary version 500. 

FIG. 9 is a flow chart illustrating operations involved in a write operation to a 
memory element by speculative thread 203 in accordance with an embodiment of tlie 
present invention. If a space-time dimensioned version 510 does not exist, the system 
creates a space-time dimensioned version 510 in speculative heap 406 (step 902). The 
20 system also updates status word 504 to indicate that speculative thread 203 has 

written to the object if such updating is necessary (step 903). The system next writes 
to space-time dimensioned version 510 (step 904). Such updating is necessary if head 
thread 202 must subsequently choose between writing to both primary version 500 
and space-time dimensioned version 510, or writing only to primary version 500 as is 
25 described above with reference to FIG. 7. 

FIG. 10 is a flow chart illustrating operations involved in a join operation 
between head thread 202 and a speculative tiiread 203 in accordance with an 
embodiment of the present invention. A join operation occurs for example when head 
thread 202 reaches a point in the program where speculative thread 203 began 
30 executing. The join operation causes state associated with the speculative thread 203 
to be merged with state associated with the head thread 202. This involves copying 
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and/or merging the stack of speculative tliread 203 into the stack of head tliread 202 
(step 1002). It also involves merging space-time dimension and primary versions of 
objects (step 1004) as wtU as possibly garbage collecting speculative heap 406 (step 
1006). In one embodiment of the present invention, one of threads 202 or 203 
5 performs steps 1002 and 1006, while the other thread performs step 1004. 

FIG, 1 1 is a flow chart illustrating operations involved in a join operation 
between head thread 202 and a speculative thread 203 in accordance with another 
embodiment of the present invention. In this embodiment, speculative thread 203 
carries on as a pseudo-head thread. As a pseudo-head thread, speculative thread 203 
10 uses indirection to reference space-time dimensioned versions of objects, but does not 
mark objects or create versions. While speculative thread 203 is acting as a pseudo- 
head thread, head thread 202 updates primary versions of objects. 

Selectively Marking Objects 

15 FIG, 12 is a flow chart illustrating the process of producing code so that 

speculative thread 203 does not mark selected memory elements in accordance with 
an embodiment of the present invention. The system starts by allowing a programmer 
to examine source code for a program that is to be speculatively executed (step 1202). 
This enables the programmer to identii^^ read operations within the source code that 

20 return values that do not affect subsequent results produced by the computation (step 
1204). For example, during a lookup mto a hash table, the lookup may involve 
reading a number of intervening entries in the hash table before finding the desired 
entry. The values read from these intervening entries do not affect the computation. 
Hence, the intervening entries do not need to be marked to trigger a subsequent 

25 rollback if the intervening entries are subsequently modified by the head thread. 

Next, the system allows the programmer to insert hints into the source code 
telling the compiler to create executable code for speculative thread 203 that does not 
to mark a given memory element during a given read operation (step 1206). In one 
embodiment of the present invention, these hints are inserted into the source code in 

30 the form of comments. These comments can be read by the compiler, or alternatively 
by a pre-processor for the compiler. 
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Next, the system compiles the source code (including the hints) into 
executable code the causes speculative thread 203 not to mark certain memory 
elements during specified read operations (step 1208). 

Note that this compilation process can talce place dynamically during program 
5 run time. Alternatively, this compilation process can take place off-line prior to 
program run time. 

FIG. 13 is a flow chart illustrating the process of compiling source code into 

executable code so that speculative thread 203 does not mark selected memory 

elements in accordance with an embodiment of the present invention. The system 
1 0 starts by compiling the source code into executable code for both head thread 202 and 

speculative thread 203 (step 1302). 

During this compilation process (or during an associated pre-compilation 

process), the system reads a comment within the source code specifying that 

speculative thread 203 is not to mark a given memoiy element during a given read 
15 operation (step 1304). In response to this comment, the system compiles the source 

code mto executable code so that speculative thread 203 does not mark the given 

memory element during the given read operation (step 1306). 

In one embodiment of the present invention this is accomplished by including 

a "look" attribute for a field within an object indicating that the field is not to be 
20 marked by speculative thread 203. At run time, speculative thread 203 examines tliis 

look attribute to determine whether a field within an object should be marked during a 

read operation. 

In another embodiment of the present invention, this is accomplished by 
creating source code for speculative thread 203 that does not mark the given memory 
25 element during the given read operation. 

Note that by reducing the number of memory elements that are marked, 
unnecessary rollbacks are eliminated, which thereby improves system performance. 
Fewer memory elements marked results in fewer loads and stores, and therefore 
improves system performance. 
30 The foregoing descriptions of embodiments of the invention have been 

presented for purposes of illustration and description only. They are not intended to 
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be exhaustive or to limit the invention to the forms disclosed. Accordingly, many 
modifications and variations will be apparent to practitioners skilled in the art. 
Additionally, the above disclosure is not intended to limit the invention. The scope of 
the invention is defined by the appended claims. 



5 
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What Is Claimed Ts: 



1 . A method for marking memory elements based upon how information 
retrieved from the memor>' elements is used in order to facilitate speculative program 
5 execution, the method comprising: 

examining source code that is to be compiled into executable code for a head 
thread that executes program instructions and for a speculative thread that executes 
program instructions in advance of the head thread; 

wherein during read operations to memory elements by the speculative thread, 
1 0 the executable code generally causes the speculative thread to update status 
information associated with the memory elements to indicate that the memory 
elements have been read by the speculative thread; 

identifying a given read operation from a given memory element, wherein a 
given value retrieved from the given memory element during the given read operation 
1 5 is not used in by the speculative thread; 

inserting at least one hint into the source code specifying that the speculative 
thread is not to update status information during the given read operation directed to 
the given memory element; and 

compiling the source code, including the at least one hint, into the executable code, so 
20 that during the given read operation, the executable code does not cause the 

speculative thread to update status information associated with the given memory 
element to indicate that the given memory element has been read by the speculative 
thread. 



2. The method of claim 1, wherein during a subsequent write operation to a 
memory element by the head thread, the executable code causes the head thread to: 

check status information associated with tiie memory element to determine if 
the memory element has been read by the speculative thread; and 

if the memory element has been read by the speculative thread, causes Ae 
speculative thread to roll back in order to read a result of the write operation. 
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3. The method of claim 1, wherein inserting the at least one hint into the 
source code involves inserting a comment into the source code that causes a compiler 
to generate executable code for the speculative thread that does not to mark the given 
memory element during the given read operation, 

5 

4. The method of claim 1, wherein compiling the source code, including 
the at least one hint, into the executable code, involves setting a look attribute 
associated with the given memory element to indicate that the speculative thread is 
not to mark the given memory element 

10 

5. The method of claim 1, wherein the executable code causes a join 
operation between the head thread and the speculative thread when the head tliread 
reaches a point in a program where tiie speculative thread began executing, the join 
operation causing state associated with the speculative thread to be merged with state 

1 5 associated with the head thread. 



6. The method of claim 1, further comprising integrating the source code 
into a system code library. 

20 7. The method of claim 1, wherein the memory elements include objects 

defined within an object-oriented programming system. 

8. A method for compiling source code into executable code for a head 
thread that executes program instructions and a speculative thread that executes 
25 program instructions in advance of the head thread, the method comprising: 

compiling the source code into executable code so that during read operations 
to memory elements by the speculative thread, the executable code generally causes 
the speculative thread to update status information associated with the memory 
elements to indicate that the memory elements have been read by the speculative 
30 thread; 
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during the compilation process, reading at least one hint inserted into the 
source code specifying that the speculative thread is not to update status information 
during a given read operation directed to a given memory element; and 

in response to the at least one hint, compiling the source code into the 
executable code, so that during the given read operation, the executable code does not 
cause the speculative thread to update status information associated with the given 
memory element to indicate that the given memory element has been read by the 
speculative thread. 

9. The method of claim 8, wherein during a subsequent write operation to 
a memory element by the head thread, the executable code causes the head thread to: 

check status information associated with the memory element to determine if 
the memory element has been read by the speculative thread; and 

if the memory element has been read by the speculative thread, causes the 
speculative thread to roll back in order to read a result of the write operation. 

1 0. The method of claim 8, wherein reading the at least one hint includes 
reading a comment within the source code. 

11. The method of claim 8, wherein compiling the source code includes 
compiling the source code dynamically as the executable code is being executed. 

12. A apparatus for compiling source code into executable code for a head 
thread that executes program instructions and a speculative thread that executes 
program instructions in advance of the head thread, comprising: 

a compiler that is configured to compile source code into executable code so 
that during read operations to memory elements by the speculative thread, the 
executable code generally causes the speculative thread to update status information 
associated with the memory elements to indicate that the memory elements have been 
read by the speculative thread; 

wherein the compiler is additionally configured to, 
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read at least one hint inserted into the source code specifying that the 

speculative thread is not to update status information during a given 
read operation directed to a given memory element, and to 

compile the source code into the executable code, so that 
5 during the given read operation, the executable code does not cause the 

speculative thread to update status information associated with the 
given memory element to indicate that the given memory element has 
been read by the speculative thread. 

10 13. The apparatus of claim 12, wherein during a subsequent write 

operation to a memory element by the head thread, the executable code causes the 
head thread to: 

check status information associated with the memory element to determine if 
the memory element has been read by the speculative tliread; and 
15 if the memory element has been read by the speculative thread, causes the 

speculative thread to roll back in order to read a result of the write operation. 

14. The apparatus of claim 12, wherein the compiler is configured to read 
the at least one hint in the form of a comment within the source code. 

20 

15. The apparatus of claim 12, wherein the compiler is configured to set a 
look attribute associated with the given memory element to indicate that the 
speculative thread is not to mark the given memory element. 

25 16. The apparatus of claim 12, wherein the compiler is configured to 

produce the executable code so that the executable code causes a join operation 
between the head thread and the speculative thread when the head thread reaches a 
point in a program where the speculative thread began executing, the join operation 
causing state associated with the speculative thread to be merged with state associated 

30 with the head thread. 
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17. The apparatus of claim 12, wherein the memory elements include 
objects defined within an object-oriented programming system. 

1 8. The apparatus of claim 12, wherein the compiler is configured to 
5 compile the source code dynamically as the executable code is being executed. 

19. A computer-readable storage medium storing instructions that when 
executed by a computer cause the computer to perform a method for compiling source 
code into executable code for a head thread that executes program instructions and a 

1 0 speculative thread that executes program instructions in advance of the head thread, 

the method comprising: 

compiling the source code into executable code so that during read operations 

to memory elements by the speculative thread, the executable code generally causes 

the speculative thread to update status information associated with the memory 
15 elements to indicate that the memory elements have been read by the speculative 

thread; 

during the compilation process, reading at least one hint inserted into the 
source code specifying that the speculative thread is not to update status information 
during a given read operation directed to a given memory element; and 
20 in response to the at least one hint, compiling the source code into the 

executable code, so that during the given read operation, the executable code does not 
cause the speculative thread to update status information associated with the given 
memory element to indicate that the given memory element has been read by the 
speculative thread. 

25 

20. The computer-readable storage medium of claim 19, wherein during a 
subsequent write operation to a memory element by the head thread, the executable 
code causes the head thread to: 

check status information associated with the memory element to determine if 
30 the memory element has been read by the speculative thread; and 
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if the memory element has been read by the speculative thread, causes the 
speculative thread to roll back in order to read a result of the write operation. 

21 . The computer-readable storage medium of claim 1 9, wherein reading 
5 the at least one hint includes reading a comment within the source code. 

22. The computer-readable storage medium of claim 19, wherein 
compiling the source code includes compiUng the source code dynamically as the 
executable code is being executed. 

10 

23. The computer-readable storage medium of claim 19, wherein 
compiling the source code, including the at least one hint, into the executable code, 
involves setting a look attribute associated witli the given memory element to indicate 
that the speculative thread is not to mark the given memory element. 

15 

24. The computer-readable storage medium of claim 1 9, wherein the 
executable code causes a join operation between the head thread and the speculative 
thread when the head thread reaches a point in a program where the speculative thread 
began executing, the join operation causing state associated with the speculative 

20 thread to be merged with state associated with the head thread. 
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(57) Abstract: One embodiment of the present invention provides a system that 
marks memory elements based upon how information reirieved from the memory 
elements affects speculative program execution. This system operates by allowing 
a programmer to examine source code that is to be compiled into executable code 
for a head thread that executes program instructions, and for a speculative thread 
that executes program instructions in advance of the head thread. During read oper- 
ations to memory elements by the speculative thread, this executable code generally 
causes the speculative thread lo update status information associated with the mem- 
ory elements to indicate thai the memory elements have been read by the speculative 
thread. Next, the system allows the programmer to identify a given read operation 
directed to a given memory element, wherein a given value retrieved from the given 
mcmor>' clement during the given read operation does not affect subsequent exe- 
cution of the speculative thread. The programmer is then allowed to insert a hint 
into the source code specifying that the speculative thread is not to update status 
information during the given read operation directed to the given memory element. 
Next, the system compiles the source code, including the hint, into the executable 
code, so that during the given read operation, the executable code does not cause the 
speculative thread to update status information associated with the given memory 
element to indicate that the given memory element has been read by the speculative 
thread. 
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